AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.687821	3.27
	2_DecisionTree	Decision Tree	logloss	0.4556	26.15
the best	3_Default_Xgboost	Xgboost	logloss	0.262653	26.93
	4_Default_NeuralNetwork	Neural Network	logloss	0.453861	17.4
	5_Default_RandomForest	Random Forest	logloss	0.388899	27.74
	Ensemble	Ensemble	logloss	0.262653	2.12

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	5

Metric details

	score	threshold
logloss	0.262653	nan
auc	0.957581	nan
f1	0.901357	0.456346
accuracy	0.889869	0.456346
precision	1	0.99804
recall	1	0.00024779
mcc	0.777034	0.456346

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.262653	nan
auc	0.957581	nan
f1	0.901357	0.456346
accuracy	0.889869	0.456346
precision	0.890713	0.456346
recall	0.912259	0.456346
mcc	0.777034	0.456346

Confusion matrix (at threshold=0.456346)

	Predicted as long	Predicted as short
Labeled as long	1710	273
Labeled as short	214	2225

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

25.2 seconds

Metric details

	score	threshold
logloss	0.262653	nan
auc	0.957581	nan
f1	0.901357	0.456346
accuracy	0.889869	0.456346
precision	1	0.998039
recall	1	0.00024779
mcc	0.777034	0.456346

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.262653	nan
auc	0.957581	nan
f1	0.901357	0.456346
accuracy	0.889869	0.456346
precision	0.890713	0.456346
recall	0.912259	0.456346
mcc	0.777034	0.456346

Confusion matrix (at threshold=0.456346)

	Predicted as long	Predicted as short
Labeled as long	1710	273
Labeled as short	214	2225

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.2 seconds

Metric details

	score	threshold
logloss	0.687821	nan
auc	0.5	nan
f1	0.710975	0.496442
accuracy	0.55156	0.496442
precision	0.55156	0.496442
recall	1	0.496442
mcc	0	0.496442

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.687821	nan
auc	0.5	nan
f1	0.710975	0.496442
accuracy	0.55156	0.496442
precision	0.55156	0.496442
recall	1	0.496442
mcc	0	0.496442

Confusion matrix (at threshold=0.496442)

	Predicted as long	Predicted as short
Labeled as long	0	1983
Labeled as short	0	2439

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

24.5 seconds

Metric details

	score	threshold
logloss	0.4556	nan
auc	0.850849	nan
f1	0.831538	0.438215
accuracy	0.802126	0.547274
precision	0.947368	0.832699
recall	1	0.12748
mcc	0.60011	0.438215

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.4556	nan
auc	0.850849	nan
f1	0.828061	0.547274
accuracy	0.802126	0.547274
precision	0.795094	0.547274
recall	0.863879	0.547274
mcc	0.598828	0.547274

Confusion matrix (at threshold=0.547274)

	Predicted as long	Predicted as short
Labeled as long	1440	543
Labeled as short	332	2107

Learning curves

Decision Tree

Tree #1

Rules

if (Total Costs > 9078.685) and (Type of Admission > 0.5) and (Total Charges > 21268.5) then class: long (proba: 85.84%) | based on 4,123 samples

if (Total Costs <= 9078.685) and (Total Costs <= 4889.275) and (CCS Diagnosis Code <= 654.0) then class: short (proba: 95.29%) | based on 3,376 samples

if (Total Costs <= 9078.685) and (Total Costs > 4889.275) and (CCS Diagnosis Code <= 253.5) then class: short (proba: 71.25%) | based on 3,033 samples

if (Total Costs > 9078.685) and (Type of Admission <= 0.5) and (Total Costs <= 23891.601) then class: short (proba: 64.61%) | based on 1,088 samples

if (Total Costs > 9078.685) and (Type of Admission <= 0.5) and (Total Costs > 23891.601) then class: long (proba: 75.26%) | based on 489 samples

if (Total Costs > 9078.685) and (Type of Admission > 0.5) and (Total Charges <= 21268.5) then class: short (proba: 54.09%) | based on 416 samples

if (Total Costs <= 9078.685) and (Total Costs <= 4889.275) and (CCS Diagnosis Code > 654.0) then class: short (proba: 65.78%) | based on 374 samples

if (Total Costs <= 9078.685) and (Total Costs > 4889.275) and (CCS Diagnosis Code > 253.5) then class: long (proba: 83.61%) | based on 366 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

16.0 seconds

Metric details

	score	threshold
logloss	0.453861	nan
auc	0.917328	nan
f1	0.861685	0.319626
accuracy	0.842605	0.319626
precision	1	0.999434
recall	1	7.34546e-53
mcc	0.682302	0.486439

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.453861	nan
auc	0.917328	nan
f1	0.861685	0.319626
accuracy	0.842605	0.319626
precision	0.836097	0.319626
recall	0.888889	0.319626
mcc	0.681215	0.319626

Confusion matrix (at threshold=0.319626)

	Predicted as long	Predicted as short
Labeled as long	1558	425
Labeled as short	271	2168

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

25.8 seconds

Metric details

	score	threshold
logloss	0.388899	nan
auc	0.910732	nan
f1	0.850787	0.4713
accuracy	0.83175	0.520931
precision	1	0.967255
recall	1	0.0596937
mcc	0.665195	0.609432

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.388899	nan
auc	0.910732	nan
f1	0.849697	0.520931
accuracy	0.83175	0.520931
precision	0.837515	0.520931
recall	0.862239	0.520931
mcc	0.659085	0.520931

Confusion matrix (at threshold=0.520931)

	Predicted as long	Predicted as short
Labeled as long	1575	408
Labeled as short	336	2103

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back